One of the major applications of neural networks is in the area of associative memory. The avalanche of intensive research interests in neural networks was initiated by the work of J. J. Hopfield, "Neural Networks and Physical Systems with Emergent Collective Computational Abilities," Proc. Nat. Acad. Sci, U.S.A., Vol. 79 p. 2254-2258 (1982), U.S. Pat. No. 4,660,166 in which the associative memory is modeled with a neural synaptic interconnection matrix and encompasses an interesting computation scheme using recursiva, nonlinear thresholding. Further investigation reported by R. J. McEliece, E. C. Posner, E. R. Rodemich, S. S. Venkatesch, "The Capacity of the Hopfield Associative Memory," IEEE Transactions on Information Theory, Vol. T-33, pp. 461-482 (1987); and B. L. Montgomery and B. V. K. Vajaya Kumar, "Evaluation of the use of Hopfield Neural Network Model as a Nearest Neighbor Algorithm," Appl. Opt. Vol. 25, pp. 3759-3766 (1986) reveals that the storage capacity of the Hopfield Model is quite limited due to the number of spurious states and oscillations.
In order to alleviate the spurious states problems in the Hopfield model, the concept of terminal attractors was introduced by M. Zak, "Terminal Attractors for Addressable Memory in Neural Networks," Phys. Lett. Vol. A-133, pp. 18-22 (1988). However, the theory of the terminal-attractor based associative neural network model proposed by Zak determines that a new synapse matrix totally different from the Hopfield matrix is needed. This new matrix, which is very complex and time-consuming to compute, was proven to eliminate spurious states, increase the speed of convergence and control the basin of attraction. Zak (1988), supra, and M. Zak, "Terminal Attractors in Neural Networks," Neural Networks, Vol. 2, pp. 259-274 (1989).
Zak's derivation shows that the Hopfield matrix only works if all the stored states in the network are orthogonal. However, since the synapses have changed from those determined by Hebb's law, Zak's model is different from the Hopfield model, except for the dynamical iteration of the recall process. The improvement of the storage capacity of the Hopfield model by the terminal attractor cannot be determined based on Zak's model.
More recently, for the purpose of comparing the Hopfield model, both including and excluding a terminal attractor, a terminal-attractor based associative memory (TABAM) model has been proposed which incorporates binary neurons into the synaptic matrix determined by Hebb's law in the same way as the Hopfield model. That work has been disclosed in a paper by H. K. Liu, J. Barhen and N. H. Farhat, "Optical Implementation of Terminal Attractor Based Associative Memory," Appl. Opt., Vol. 31, pp. 4631-4644, Aug. 10, 1992, which by this reference is incorporated herein and made a part hereof. Among the several techniques proposed for optical implementation of the terminal attractor, the most important includes the application of the inner-product approach first proposed by S. Y. Kung and H. K. Liu, "An Optical Inner-Product Array Processor for Associative Retrieval," Proc. SPIE, Vol. 613, pp. 214-219 (1986) and later fully implemented by H. K. Liu, U.S. patent application Ser. No. 07/880,210 titled "Optical Inner-Product Neural Associative Memory," which by this reference is incorporated herein and made a part hereof. Also included among the several techniques proposed for optical implementation of the terminal attractor is the exclusive-or (XOR) operation of the liquid crystal television spatial light modulator (LCTV SLM) described by H. K. Liu and T. H. Chao, "Liquid Crystal Television Spatial Light Modulators," Appl. Opt., Vol. 28, pp. 4772-4780 (1989).
The complexity of the optical implementation of a TABAM is discussed in the 1992 paper by Liu, Barhen and Farhat. In general, optical implementation of subtraction increases the complexity of a TABAM system. Consequently, a unipolar neuron model, which has only 1 and 0 as binary states, instead of 1, 0 and -1 would be more suitable for implementation of a TABAM, but that would require optical implementation without subtraction or negative numbers normally encountered in a TABAM.